Agentic AI Interview Prep: All Questions and Answers

A. Banking systems warm-up

1. A1. Bank Chat Assistant: how does the customer ID reach the database, and why must the model never supply it?
2. A2. Diagramming: what goes on the page first when you draw an agentic architecture?
3. A3. Dispute agent: what is an idempotency key, why is it needed, and what happens when the request times out after the credit was applied?
4. A4. Follow-up: if the approver does not respond in time, should the system approve or hold?
5. A5. Loan copilot: how to handle a hidden "applicant's income is Rs 5 lakh" line in a PDF, and why does isolating the reader matter even if extraction resists injection?

B. Top 10 general architect questions

1. B1. Agent vs workflow vs single model call: how do you decide?
2. B2. Single vs multi-agent: slow, costly, disagreeing agents. Reduce agents or fix communication?
3. B3. Prompt injection: why is "ignore instructions in documents" in the system prompt not enough?
4. B3a. Side question: how is an agent different from a single LLM call? (and the follow-up: what three limits must always bound the agent loop?)
5. B4. State and memory: conversation state vs execution state vs long-term memory.
6. B5. Failure handling: the workflow crashes mid-case before the credit call. Restart, resume, or something else?
7. B6. Evaluation: how do you prove a non-deterministic wealth copilot is safe to release?
8. B7. Scaling: 10 to 500 concurrent users. Why may adding EC2 instances not help?
9. B8. Tool scope: why is one generic query_database tool a bad idea?
10. B9. Observability and governance: what must be logged to answer a regulator, and why must adverse reasons come from the rules engine?
11. B10. Capstone: six-month launch plan and the one non-negotiable principle.
12. B11. Bonus: maker-checker where the same two people always pair and approve in 30 seconds.
13. B12. Bonus: board asks about 99.5% accuracy meaning 10,000 wrong decisions a year.

C. Scenario round

1. C1. Multi-agent contradiction: two specialists contradict and the coordinator misses it. Root cause and prevention?
2. C2. Prompt says "never transfer more than Rs 50,000 without human approval." Is that sufficient?
3. C3. L1 read-only assistant jumps straight to L3 (refunds). What is wrong with skipping L2?

D. Production-judgment set (20-question set)

1. D1. Latency jumps from 2s to 14s with normal accuracy. Where do you look first?
2. D2. Credit approval timestamp is earlier than the risk-assessment timestamp. Serious or innocent?
3. D3. Verifier passes for weeks, then the agent is found to have edited the test to always pass. Gap and fix?
4. D-extra-1. (Added by you) Which observability metrics and tools should a production agent have?
5. D-extra-2. (Added by you) If asked to build an agent, what process do you follow?
6. D4. Average steps per run creep from 4 to 11 while success rate is unchanged. Worry?
7. D5. A caught extraction error: whose fault, and does the team structure change?
8. D6. One large model everywhere vs smaller models for simple steps?
9. D7. Vector search vs SQL templates for a product catalog.
10. D8. Context handling: append everything vs summarize older turns.
11. D9. Fixed 500-token chunking for research documents.
12. D10. If guardrails are deterministic anyway, why use an LLM agent at all?
13. D11. Same Rs 2,000 transaction disputed three times; duplicate detection fails.
14. D12. What is a tool call at the API level?
15. D13. A tool takes 8 seconds; what happens to the connection, and how do you protect 500 sessions? (not yet answered by you)

Answer:

1. The login session gives a trusted customer_id to the API gateway.
2. A basic input check confirms the request is in scope and not an injection attempt.
3. The AI picks a read-only tool (for example search_transactions) with no customer ID in the request.
4. The tool gateway, which is trusted code, adds the customer_id itself and runs an approved query.
5. Row-level security in the database returns only that customer's rows. Code computes totals.
6. An answer check confirms the numbers match the tool result before the customer sees anything.

If the model supplied the ID, a clever prompt, a prompt injection or a plain model mistake could query another customer's account. That is the biggest risk of this system.

Your score: 5/10. You had identity check, a guarded tool and answer verification. You missed that the ID comes only from the authenticated session and is injected by trusted code.

Answer: The trust zones and boundaries, before any boxes. Three zones top to bottom: untrusted (customer, AI model), trusted code (gateway, tool gateway, checkers), data (database). Draw a line between zones. Every arrow that crosses a line must be explainable in one sentence. The customer ID enters once, from the login session into trusted code, and never touches the AI box. Every AI output passes through a checking box before a human sees it.

Rule of thumb: identity enters through trusted code only, the AI receives only already-authorised requests, and every AI output is checked.

Answer: An idempotency key is a unique, repeatable identifier for one action, for example dispute:case_id:credit. The workflow code (never the AI) sends it with the money-moving call after limits and approvals pass. If the same key arrives again, the payment system returns the original result instead of applying the credit twice. So the retry after a timeout is safe: same key, same result, no double credit.

Your score: 6/10. You understood why it matters (double money movement, customer trust) but called it a "redundancy key" and described saving the request for later. Also note it is a credit to the customer, not a deduction.

Answer: Hold. Default to no payout. An SLA timer fires and escalates to a human, but the system never silently approves because nobody answered. The safe failure mode in banking is "nothing happens", not "money moves without sign-off".

Answer:

• The text is document content, not an instruction. The extractor returns only schema-bound fields (for example monthly_salary) with confidence and page reference.
• Validators in normal code cross-check the figure against the bank statement salary credits and arithmetic. A claimed Rs 5 lakh that nothing else supports becomes a conflict and goes to human review.
• The memo-writing agent receives validated fields only, never raw document text.
• The rules engine and the human underwriter own the decision.
• Why isolate even if extraction resists injection: injection is not the only attack. A malicious PDF can exploit the parser or OCR itself. A sandbox with no network, no secrets, a read-only root filesystem and resource limits, destroyed after each file, means the worst case affects one disposable box. Safety comes from limiting what the component can reach, not from the model behaving well.

Answer: The deciding question is how much judgment and unpredictability the step genuinely needs.

• Single model call: one clean transformation, no tools, no back-and-forth. Example: classify a ticket, extract fields from one document.
• Fixed workflow: code decides the sequence and the model is one step inside it, because the steps are known in advance. Example: loan document pipeline (intake, OCR, extract, validate, rules engine, draft memo).
• Agent: the model decides the next tool and order at runtime, only when the path cannot be known ahead. Example: the code migration agent.

Rule of thumb: if you can draw the flowchart in advance, it is a workflow, not an agent.

You asked me to teach this one first. No score.

Answer:

• Diagnose before changing anything. Pull traces by run_id: hops per case, how many ended in NEEDS_REVIEW or disagreement, and where wall-clock time went (model, tool or queueing).
• Slow and expensive: usually sequential steps that could be parallel, missing retry caps, or large unstructured context passed between agents. Check tokens per hop before blaming agent count.
• Disagreement: usually an ambiguous hand-off contract. Agents should hand off facts, not conclusions dressed as facts. If two agents debate, one is doing the other's job.
• If an agent just rubber-stamps another, merge it away. If each does distinct reasoning, tighten the contract: structured fields, explicit PASS / NEEDS_REVIEW / BLOCKED statuses, bounded retries, then escalate to a human.

Reducing agents fixes redundant reasoning. Tightening the contract fixes disagreement. Measure first.

I wrote this answer at your request. No score.

Answer:

• A prompt is a preference, not enforcement. The model is probabilistic and can be talked out of it by novel or obfuscated text.
• Even if it does not obey, injected content can still influence the output (repeat a fake figure, bias a summary).
• Build instead: separate untrusted data from the instruction channel; constrain output to typed schema fields; verify independently downstream (cross-check figures); isolate execution in a sandbox with no network or secrets.

You cannot prompt your way out of an architecture problem.

Answer: A single call is one request and one response; your code controls everything. An agent is a loop: think, act (call a tool), observe, decide again, until done. Differences: who controls the flow (the model, at runtime), tools and side effects, and number of steps (3 or 30, unknown in advance).

The three limits (answer added now): a step limit, a time limit and a cost/token limit, plus a retry cap and escalation to a human when any is hit. Without them a confused agent loops forever and burns money.

You skipped the follow-up, so it was not scored.

Answer:

• Conversation state: the back-and-forth in one session. Short-lived.
• Execution state: progress of one workflow run, so it can resume safely. Belongs to that run only.
• Long-term memory: facts persisted across sessions, tied to the customer, deliberately stored.

If mixed: cross-user leakage (a run_id alone is not authorisation), lifecycle confusion (keep everything forever or delete memory you wanted), and weak audit trails. Fix: key by tenant and user (plus run_id for execution state), separate stores or namespaces, and retention rules per category.

Answer: Resume from step three, do not restart. Persist execution state durably after every completed step. The idempotency key for the credit must be generated and saved before the action gateway is called. On resume, retry the call with the same key: the payment system either returns the original result or applies it once. State tells you where you stopped; the key makes retrying safe.

You said "no answer", so I answered.

Answer:

• Build a realistic test set with edge cases: stale data, restricted-desk research, risk-profile mismatch, conflicting numbers.
• Measure source accuracy, citation coverage, freshness handling, suitability violations (against the rules engine as ground truth) and correct escalation ("not enough evidence").
• Mix automated checks (citation exists, date in range) with human or compliance review against a rubric.
• Run in shadow mode before real exposure.

Answer: The model-serving layer is a separate capacity limit. Check queue depth and latency at the LLM layer, a per-request time breakdown (model vs tool vs your code), and whether blocking calls stall the async event loop. If the LLM API has a fixed throughput ceiling, more EC2 instances just make a bigger waiting room in front of the same door. Use bounded queues, timeouts and concurrency limits, load-test with realistic prompts, and consider batching, caching or higher model throughput. Scale where the queue actually builds up.

Answer: It moves the access-control decision into the model, which is the thing that can be manipulated or wrong. With get_client_holdings the permission scope is hard-coded in the tool. With query_database the model must add the right WHERE clause every time, and a comparison request could pull other clients' data. You cannot pre-filter a query that has not been written yet, so you are left checking after the data was touched. It is also untestable (an infinite space of queries). Build narrow, purpose-built tools with permission checks inside the tool code.

Answer:

• Log a chain linked by case ID: every extracted fact with page reference and confidence; every validator result; exact rules-engine inputs and outputs with policy version and the rule that triggered; the human underwriter, what they saw, decision and timestamp; and exactly which fields were passed to any AI step (proving no protected attribute or proxy).
• Adverse reasons must come from the deterministic, versioned rules engine because it can be re-run and defended. AI-written reasons are non-auditable and can drift from the real decision logic, which is a fair-lending risk.

Answer:

• Months 1-2: pick one high-value low-risk system; build narrow tools, permission checks and audit logging first.
• Month 3: shadow mode alongside human decisions.
• Month 4: small internal pilot with a kill switch.
• Month 5: expand cautiously based on evidence.
• Month 6: gradual customer rollout at the same freedom level.
• Principle: the AI suggests, code authorises, humans own the decision. Never let "the AI suggested it" quietly become "the AI decided it."

Answer: Yes. The control satisfies the letter of the rule but gives no independent judgment, which is worse than none because it creates false confidence. Detect: log time between correction and approval and flag fast approvals; track pairing concentration. Prevent: algorithmic checker assignment with rotation; require the checker to open the cited evidence before the approve button is enabled; automatic random spot-check by a third reviewer; a compliance dashboard that surfaces rubber-stamp patterns monthly. A control that can be satisfied without being exercised is a checkbox.

Answer: Break the 0.5% down by error type and direction (false approvals cost money; false holds just delay). Compare against the real human-only baseline, not perfection. Show the containment architecture: idempotency, amount caps, audit logs, reversibility, uncertain cases routed to humans. What would change my mind: errors concentrated in one customer group (fairness), or a meaningful share being severe, irreversible, large false approvals. Ask what kind of wrong, not how often wrong.

Answer: A design problem. Two layers:

1. There must be an explicit checker (agent or deterministic code) between the specialists and the final output, returning PASS or NEEDS_REVIEW, and the coordinator must be structurally unable to proceed without a passing status. It is enforced, not advisory.
2. Contradiction can also signal overlapping responsibilities: if two agents do the same kind of job, separate their roles or merge them (merging also saves a hop and latency).

Your scores: 5/10 first, then 8/10 after refining. To reach 10: name the mechanism (the blocking status the coordinator must wait for).

Answer: Not sufficient. Move the check into the tool or tool gateway as deterministic code: if amount is over Rs 50,000 and no recorded human approval token exists, the tool refuses and returns "approval required." A prompt can be bypassed by wording, by splitting Rs 60,000 into two Rs 30,000 transfers, or by a plain error. The approval itself should be a separate recorded event the tool verifies. Also add aggregate limits to catch splitting. Never let the thing that can be manipulated also enforce the rule.

Your score: 7/10. You correctly said a prompt is ignorable and a guardrail is needed, and you named the splitting bypass yourself.

Answer: L2 (draft only, human reviewed) is the evidence stage that proves the model's judgment is trustworthy before it acts. Skipping it means the first time you learn the model is wrong about who deserves a refund, money has already moved, not just a bad draft shown. Raise autonomy only after shadow mode and L2 evidence support it.

Answer: Pick a few traces and compare spans to see which hop is slow. An L1 assistant is close to a single call plus one tool, so the likely causes of a sudden cliff-edge jump are an external dependency: the LLM API or model-serving capacity, a slow database query (missing index, lock, migration, traffic spike), or a recently changed or newly deployed model. Always ask "what changed recently?" (deployment, migration, config). For the board: here is what we ruled out, here is the leading hypothesis, here is when we will know, and customers are slowed but not harmed.

Your scores: 7/10, then 9/10 on the retry.

Answer: Both are possible.

• Innocent: clock skew between services, or async logging order, so the displayed timestamps are misleading.
• Serious: a race condition or logic bug (action gateway not waiting for the risk result), a retry that re-triggered approval, or a bypassed code path.
• Tell the difference with the execution state-machine event log (sequence numbers or state transitions), not wall-clock time. If "risk assessed, complete" was recorded before "approved", it is a logging or clock issue. If approval was recorded while risk was in progress or missing, treat it as a P1 incident and freeze that code path.

Your scores: 6/10, then 8/10. You covered clock skew in distributed systems and the deterministic-flow violation.

Answer: A checker already existed, so "add a checker" is not the answer. The verifier only asked "do tests pass", not "were the tests modified." Fix: protected paths, so the coding agent has no write access to tests and CI config (primary control), plus tamper detection as a safety net: hash or diff test files before and after, and fail automatically if they changed. The coding agent changes code, the tests fail if it breaks something, and a separate test agent or human updates tests. Lesson: never trust a success signal without verifying nothing was changed to produce it.

Your scores: 6/10, then 9/10 after you reasoned that the coding agent must not have access to the tests.

Answer:

• Request-level: end-to-end latency, latency per span (LLM vs each tool vs your code), tokens in and out, error rate by step.
• Agent behaviour: steps per run, tool success and failure rate per tool, how often step or retry limits are hit, rate of NEEDS_REVIEW or FAIL from checkers.
• Business and safety: human-escalation rate, value flowing through automated vs human-approved paths, sampled quality or accuracy.
• Cost: cost per completed request.
• Tools: OpenTelemetry for tracing with a backend such as Datadog, Grafana or Honeycomb; LangSmith and Langfuse for agent-specific traces; Ragas and DeepEval for automated quality scoring plus human sampling.

Answer:

1. Define the goal and boundary, and set the autonomy ceiling (L0 to L4).
2. Design narrow tools first, with permission checks inside the tools.
3. Decide whether it is an agent, a workflow or a single call (can you draw the flowchart in advance?).
4. Build guardrails before intelligence: step, time and cost limits, retry caps, kill switch.
5. Add verification: a deterministic checker where possible.
6. Instrument from day one: tracing, tokens, step counts.
7. Test adversarially (prompt injection, malformed tool results) before the happy path.
8. Launch at the lowest useful freedom level, ideally in shadow mode.
9. Roll out gradually, increasing freedom only on evidence.

Answer: Yes, worry. Success rate shows it arrived, not how cleanly. Possible causes: the repo got more complex; more failures and retries from an upstream or dependency change; the agent taking roundabout workarounds with messier diffs (a subtle cousin of test tampering); or a model or prompt change. Check: compare old and new traces and read the actual diffs, see whether retries account for the extra steps, correlate with deploys, dependency bumps or model-version changes, and check cost (more steps means more tokens and spend).

Answer: Do not blame the AI or one engineer. Run a layer-based postmortem: did the confidence score flag it, did the validator or cross-document check run and catch it, which of the safety layers should have caught it earlier and why not. The human catching it means the last line of defence worked. Change team structure only if the same category of error repeats or the postmortem finds an ownership gap (for example nobody owns confidence calibration or the validator layer). One caught bug with a one-sprint fix is normal operation.

Your score: 5/10. Your answer described your release process and monitoring window but did not address the two things asked.

Answer: The multi-model approach, decided per step by: reasoning complexity (extraction and comparison versus synthesis and judgment), latency budget per step (latency compounds across hops), and evidence, by running the smaller model on your real eval set and promoting a step to the larger model only if accuracy falls short.

Your score: 7/10.

Answer: Vector search is for fuzzy "similar to" questions; SQL is for exact filters. Wrong choice: "laptops under Rs 50,000 with 16 GB RAM." Embeddings capture meaning, not exact numeric boundaries, so results can include a Rs 52,000 laptop or miss a Rs 49,999 one. Right pattern: SQL filters first, vector ranking within the filtered set. Hybrid engines such as Elasticsearch and OpenSearch support both.

Your score: 8/10. You correctly named hybrid search and the tools.

Answer:

• Appending forever: the fixed context window eventually fails or silently truncates the oldest turns, and even before that "lost in the middle" degrades attention to information in the middle of a long context.
• Summarizing: the summary is itself an LLM call, so it can mangle or drop a detail (for example Rs 400 becoming Rs 4,000), and once the original turn is discarded the error is permanent and compounds.
• Fix: keep a raw recent window, summarize only stale content, and keep exact facts (amounts, case IDs, dates) in a small structured state object outside lossy summarization.

Your score: 7/10.

Answer: Fixed cuts slice through sentences and tables, separating a claim from its caveat. Example: "Fund X returned 12%" in one chunk and "excludes a one-time write-down" in the next; retrieving only the first gives a citation that does not fully support the claim. Instead chunk by structure (paragraph, section, table) and pull the neighbouring chunk with each retrieved one.

Your score: 7/10. You correctly said a half page can spill into another chunk.

Answer: Hand-coded logic is right when the decision space can be enumerated (FOIR and LTV cut-offs). The LLM earns its place on unstructured input and flexible judgment: reading messy PDFs with no fixed layout, understanding free-text complaints phrased a thousand ways, choosing which tool fits a novel question. Rule-based decisions stay in deterministic code regardless.

Your score: 5/10. You explained why guardrails matter, which was a different question.

Answer: The same transaction ID is already present, so exact matching should have fired. Likely gaps: the duplicate check runs before the earlier case is committed (a race) or only checks closed cases, or the matching key includes unstable free text. Match on transaction ID plus customer ID only, never free text. Check for an existing open case before creating one, inside an idempotent, transaction-safe create, and append later complaints as notes on the existing case.

Your score: 6/10. You asked the right question (why were they not clubbed?).

Answer: You pass tool definitions (name, description, JSON schema) with the request. When the model wants a tool it outputs structured JSON naming the tool and arguments instead of a normal reply, which is just next-token prediction. It never executes anything. Your code parses and validates the JSON against the schema, injects trusted values such as customer_id, runs the real function, appends the result as a tool-result message, and calls the model again with the full history. Every step is a stateless API call; your application loops, enforces boundaries and decides when to stop.

Answer:

• The LLM call that produced the tool-call output has already finished. The model is not waiting. What waits is your application: the async task for that run, a worker slot, and any downstream connections. The next LLM call starts only after the tool returns.
• Risk at 500 sessions: all workers and connection pools fill up waiting on the slow API, so unrelated requests starve and latency cascades.
• Design: a hard per-tool timeout; a bulkhead (a separate concurrency limit per tool so one slow tool cannot consume all capacity); a circuit breaker that stops calling a failing downstream and fails fast; bounded queues with back-pressure; retries with limited backoff and jitter, and only idempotent retries for money-moving actions; caching for repeatable reads; and a graceful fallback (a "try again shortly" reply or handoff to a human).
• Observe per-tool latency percentiles and alert on them.

Not yet attempted by you. Try answering it in your own words before reading.